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ABSTRACT 

Different ways that dependent variables can cause 
multivariate significance are illustrated. It is argued that 
univariate tests should not be used as follow-up procedures to 
identify the variables that can contribute to multivariate 
significance. Methods that can be used to determine the contributions 
individual variables mafce to multivariate significance are discussed. 
Two-group multivariate analysis of variance (MANOVA) with two 
dependent variables is described; a significant multivariate test 
under three conditions of univariate significance is considered: (1) 
with both dependent variables significant; (2) with one dependent 
variable significant; and (3) with both dependent variables not 
significant. The relationships among the variables in these 
conditions yield 11 different situations. The data illustrate: 
multivariate significance can be caused under a variety of 
relationships; it is improper to interpret univariate analysis of 
variance results to find the causes of multivariate significance. The 
use of discriminant analysis and stepwise discriminant analysis is 
demonstrated as a valuable tool in considering the contributions of 
dependent variables to multivariate significance. Three tables 
illustrate the 11 situations, and Appendix A details the process used 
to create each situation. (SLD) 
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Causes Of Multivariate Significance 
In A Multivariate Analysis Of Variance 

Objectives 

The purpose of this paper is threefold: first, to illustrate the different ways that 
dependent variables can cause multivariate significance; second, to reinforce the 
position that univariate tests should not be used as follow-up procedures to identify 
the variables that contribute to multivariate significance; third to discuss methods 
that can be used to discern the contributions that individual variables make to 
multivariate significance. 



Perspectives 

Many of the most popular multivariate analysis of variance textbooks in the social 
sciences indicate that the importance of a variable to multivariate significance can 
be found by selecting the dependent variables that have significant univariate tests. 
Quotes from a few of these books are given below. 



The DVs [dependent variables] that have significant univariate F's are the 
important ones, and they can be ranked in importance by strength of 
association (Tabachnick & Fidell, 1989, p. 399). 

Wilks' A is significant at .05 (F ■ 4.361, p < .000), indicating that the three 
groups differ on the set of 4 variables. The univariate F's are all significant at 
the .05 level, showing that all of the variables contributed to the multivariate 
significance (Stevens, 1986, p. 160). 

When the hypothesis of no difference is rejected, it is often informative to 
examine the univariate test results to get some idea of where the differences 
may be (Norusis, 1985, p. 207). 

We have found that students and faculty alike will look at the univariate tests that 
follow the overall multivariate test in a multivariate analysis of variance 
(MANOVA) to find what raused the multivariate significance. Although we feel 
that the above authors are well aware of the problems associated with interpreting 
multivariate significance on the basis of univariate significance, statements like 
those above encourage researchers to make improper interpretations of 
multivariate significance. 

The focus of this paper is on the circumstances under which a set of dependent 
variables cause a given multivariate test to be significant. In the process of 
examining these circumstances it is obvious that the preceding quotations are not 
necessarily true. Follow-up procedures that can help educational researchers 
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identify the causes of multivariate significance are suggested. 



Methods 

Parti: Two Group MANOVA 

The basic process followed was to initially consider the two group MANOVA with 
two dependent variables and a significant multivariate test under three conditions 
of univariate significance: both dependent variables significant (condition 1), one 
dependent variable significant (condition 2), both dependent variables not 
significant (condition 3). Within each condition different relationships among the 
dependent variables and between the dependent and independent variables were 
considered. This yielded a total of 11 different situations that are discussed and 
illustrated. 

The formulas used are presented in Appendix A. Although there are other 
situations that could have been presented, we feel that the 11 we selected provide 
excellent support for our arguments. 

The analyses in this part were performed using SPSSX (SPSS Inc., 1988) using 
summary (matrix) input and the programs MANOVA and REGRESSION. The 
follow-up procedure considered was discriminant analysis. (Step-Down analysis 
was also considered as part of the output, but was not as informative as the 
discriminant analysis.) The results of the analyses are given in Appendix B which 
is available from the first author. 

Definitions 

What do researchers mean when they say that a dependent variable caused 
multivariate significance? We are not sure, but the following definitions are what 
we mean by the terms caused and contributed to multivariate significance. 

A dependent variable is said to have caused a multivariate test to be 
significant when the part of the variable left after the effects of the other 
dependent variables in the analysis have been partialed from it would by itself 
yield a significant multivariate test. 

A dependent variable is said to have contributed to the significance of a 
multivariate test when the part of the variable left after the effects of the other 
dependent variables in the analysis have been partialed from it is 
determined, on the basis of a rationale provided by the researcher, to be 
important in identifying group differences. 

Part 2: Factorial MANOVA 

In Part 2 the material developed in Part 1 was generalized to two groups and three 
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dependent variables and to multiple groups and multiple dependent variables. 



Part 1 Results: Two Group MANOVA With Two Dependent Variables 

For all of the conditions for the two group MANOVA with two dependent variables 
we found that the canonical discriminant analysis results (Bardkowski, 1983, p. 
47; Green, 1979), and not the univariate tests alone, clearly illustrated the 
contributions of the dependent variables to multivariate significance. Our 
criterion was the relative sizes of the standardized discriminant function 
coefficients. We round that the relative sizes of the standardized discriminant 
function coefficients allowed us to determine the relative contributions of the 
dependent variables to multivariate significance. We also found that the 
discriminant function variable correlations (i.e., the discriminant structure 
coefficients) mirrored the univariate test results. 

Condition 1: Multivariate Significance And Both Univariate Tests Not 
Significant 

If one is to interpret the causes of multivariate significance by considering the 
significant univariate tests (as is suggested by the textbook authors at the 
beginning of this paper), what does one do when the multivariate test is significant 
but both of the univariate tests are not significant? As it turns out; this case is the 
easiest to interpret because with only two dependent variables multivariate 
significance must have been caused by contributions of both dependent variables 
in some way. We can say this because if only one variable was causing 
multivariate significance, that variable would have to be significant at the 
univariate level. 

Table 1 illustrates four situations for condition 1 (multivariate significance and 
both univariate tests not significant). In the first situation both dependent 
variables were uncorrected and both dependent variables were related to the 
independent variable. In this situation both variables independently "gang up" to 
cause multivariate significance. 

In the second situation the dependent variables were correlated. Here, the first 
dependent variable was correlated with the independent variable and the second 
dependent variable was not correlated with the independent variable. However, 
the first dependent variable acted as a suppressor variable to the second dependent 
variable in the relationship between the second dependent variable and the 
independent variable. In the example for this situation the second variable shows 
ap-value of 1.0, but when the first variable was partialed from the second variable 
what was left of the second variable "ganged up" with the first variable to cause 
multivariate significance. 

The third and fourth situations were variations of the first two situations. In the 
third and fourth situations both dependent variables were correlated, and both 
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Detection of the Correlations Among the Variables and of Example Discriminant Analysts Outout for Condition i • 
Overs* Multivariate Significance (p < .05) And Both Untvartate f Tests Mot Slgtirticam^V^OS)^ conowon i . 

S2!S2KS^ nTh# Exptonatton Example Discriminant 

Depend*** Variables ^pendem Vartabto (X) And Function Coefficient 

( 142 ) The Dependent Variables (1A 2) variable — CaafHctonti 





Situation 1 








p» - 0.0 


p„ # 0.0 and p^ * 0.0 


Both dependent variables 


1 


.708 


(r u = 0.00)* 




are related to essentially 


2 


.706 


(r„ » 266; r u =.254) 


independent parts of the 








independent variable 








Situation 2 








ft 4 o n 
Pit * u u 


p tl * u.U and *» 0.0 


One dependent variable, 1, 


1 


1.412 


\T lt - . #UO; 


(r tl = .255; r^= 0.00) 


is arbitrarily chosen as a 


2 


-.997 


suppressor variable. When 








1 is partialed from 2 the 










relationship between 2 and x 










is increased. 








Situation 3 








p lf *0.0 


p„ * 0.0 and p i2 / 0.0 


Both dependent variables 


1 


.651 


(r w s.l00) 




contribute to multivariate 


2 


.697 


(r il = .260; .274) 


significance. 






Situation 4 








p lt *Q.O 


p„ * 0.0 and p rf * 0.0 


Both dependent variables 


1 


.730 


(r 18 = -.100) 


(^ = .250; r ia =.262) 


contribute to multivariate 


2 


.760 


significance, but 1 also acts 








as a suppressor in the 










relationship between 2 and x. 







1 Standardized canonical discriminant function iwilicifiiu. 

'Example values utt-d u, illustrate a giv, n situation and Inmate the discriminant function coefficients are given in narvntheses. 
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dependent variables were related to the independent variable. In the third 
situation both variables "ganged up" to cause the multivariate significance. In the 
fourth situation both dependent variables contributed to multivariate significance, 
but the first dependent variable also acted as a suppressor variable in the 
relationship between the second dependent variable and the independent variable. 
Therefore, in the fourth situation the second dependent variable contributed to the 
overall multivariate significance in a stronger way than if the first dependent 
variable was not present 

Discriminant Analyses: Standardized Discriminant Function Coefficients. For all 
of the examples used to illustrate the preceding situations the discriminant 
function coefficients were relatively large (e.g., greater than .65) for both dependent 
variables. These coefficients are partial coefficients in that they axe derived so that 
the contribution of the other dependent variable is partialed (removed) in their 
formation. In this way the discriminant function coefficients take into 
consideration the relationships both between the dependent variables and between 
the dependent variables and the independent variable. Therefore, when these 
coefficients are both large this indicates that both dependent variables contributed 
to the overall multivariate significance. 

Discriminant Analyses: Discriminant Function Variable Correlations, In the 
examples used for this condition the discriminant function variable correlations 
conveyed information similar to that provided by the univariate tests. In general 
the smaller the p-value for a univariate test the larger is the corresponding 
discriminant function variable correlation. For example, in the data analyses for 
situations 1 and 3 the univariate tests had relatively low p-values (e.g., .054 £p £ 
.075) and the discriminant function variable correlations were relatively large 
(e.g., .706 ^ r £ .762). In situation 2 the univariate test for the first dependent 
variable yielded a low p-value (p < .074), but the univariate test for the second 
dependent variable yielded a large p-value (p < 1.0). These univariate results were 
conveyed by the discriminant function variable correlations in situation 2 where 
the discriminant function variable correlation was relatively large (.708) for the 
first dependent variable and was zero for the second dependent variable. 

Condition 2: Multivariate Significance And One Univariate Test 
Significant And One Univariate Test Not Significant 

The four situations within this condition are illustrated in Table 2. According to 
our textbook authors, in this condition we would attribute the significance of the 
multivariate test to the single variable whose univariate test is significant. Of the 
four situations wt chose to illustrate this condition, in only one of them (situation 
5) would the textbook authors be correct according to our definition of caused. 

From our examples you can see that a basic question is: "When does the 
contribution of the nonsignificant dependent variable to overall multivariate 
significance become an important contribution?". (Perhaps this question could be 
answered by comparing the correct classifications of the discriminant function 
based on only the significant dependent variable with the classifications found 
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Table 2 

Description of tut Correlations Among the Variables and of Example Discriminant Analysis Output for Condition 2: 

Oven* Multivariate Significance (p < .05), First Univariate t Test Significant (p < .05) And Second Univariate f Test Not Significant (p > .05) 

Relationship Between The Relationship Between The Explanation Exempli Discriminant 

Dependent Variatott* Independent variable (X) And Function Coefficients 

(1*2) The Dependent Variables (1 & 2) Variable Coefficient 1 



Situation 5 



P M -o.O 



(r tt = 0.00) 



Pit* 00 



(r u =.600) 



(r lt =.260) 



p„ * 0.0 and » 0.0 
(r ml =.350; r^O.00) 

p 4l * 0.0 and p^ * 0.0 
(r Bl = .300; r l2 = .199) 



Situation 6 



Situation 7 



p f| # 0.0 and p, 2 - 0.0 
(r,, = .290; r i2 =0.00) 



Situation 8 



p^ t 0.0 and p ti| * 0.0 
(r„ = .300; r^s .268) 



The first dependent variable 
is responsible for multivariate 
significance. 



Both variables contribute 1 
to multivariate significance 2 
but variable 1 contributes 
more than variable 2, 



Both dependent variables 1 
contribute to multivariate 2 
significance. Variable 1 acts 
as a suppressor in the 
relationship between 2 and x. 



Both dependent variables 1 
contribute to multivariate 2 
significance. 



1.000 
.000 



.840 
.543 



1.250 
-.750 



.695 
.566 



1 Standardized canonical discriminant function ctH'tficitmts. 

"'Kxainple values usst-d to liiu.ttraU- a j;iv«-ii .Mtuatioii and u» ciViilc i\w discriminant function codlictenU are given in paivnlln-srs. 
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using both dependent variables.) 

In situation 5 the dependent variables were uncorrelated and only the first 
dependent variable was related to the independent variable, the second dependent 
variable was not This was the only situation within condition 2 where one could 
clearly decide on the basis of the univariate tests that the first dependent variable 
was causing the multivariate significance. This same information was conveyed 
by the discriminant function coefficients and the discriminant function variable 
correlations. Indeed in this case the discriminant function coefficients and the 
discriminant function variable correlations were equal. Both the discriminant 
function coefficient and the discriminant function variable correlation were 1.00 
for the first dependent variable and 0.00 for the second dependent variable. 

Situation 6 is similar to situation 5 in that the two dependent variables are 
uncorrelated, however in situation 6 both dependent variables were related to the 
dependent variable and both may be described as contributing to the multivariate 
significance. In the example for this case the discriminant function coefficients 
(.840, .543) and the discriminant function variable correlations (.840, .543) were 
again equal, and their magnitudes indicated that both dependent variables were 
contributing to the overall multivariate significance. 

In situation 7 both dependent variables were correlated. Here, the first dependent 
variable was significantly correlated with the independent variable and the second 
dependent variable was not correlated with the independent variable. However, 
the first dependent variable acted as a suppressor variable to the second dependent 
variable in the relationship between the second dependent variable and the 
independent variable. In the example for this situation if you only examined the 
univariate tests you would surely think that only the first dependent variable 
contributed to the multivariate significance because the p-value for the first 
univariate test was significant (p < .041) while the p-value for the second univariate 
test was 1.000. Here, the discriminant function coefficients, with each dependent 
variable having relatively large coefficients (1.250 and -.750), indicated that both 
dependent variables contributed to the overall multivariate significance. 

Situation 8 is similar in description to that of situation 3 and we could have created 
a situation similar to that of situation 4. The basic idea of these situations is that 
both variables contributed to multivariate significance even though only one of 
them is significant at the univariate level. 

Condition 3: Multivariate Significance And Both Univariate Tests 
Significant 

Three situations were considered in Table 3 under the condition where the 
multivariate test is significant and ooth univariate tests were significant. In two 
of the three situations (9 and 10) the univariate tests do reveal the contributions 
made to overall multivariate significance. However, in the last situation (11) only 
one dependent variable was contributing to the multivariate significance. Another 
situation, similar to situation 7 could have been created to consider the situation 



Table 3 

Description of the Correlations Among the Variabtos and of Example Discriminant Analysis Output (or Condition 3: 
Overall Multivariate Significance (p < .05) And Both Univariate t Tests Significant (p<.05) 



Relationship B e tw e en The Relationship Between The 
Dependent Variaoies independent Variable <X) And 

(1ft 7) The Dependent Variables (1 & 2) 



Explanation 



Example Discriminant 
Function Coefficients 
Variable Coefficient' 



Situation 9 



P„-0.0 

(r lt = 0.00) J 



(r w u .250) 



Pa* 0.0 
(r lt = .756) 



p„ * 0.0 and p u * 0.0 
(r tl = .280; ^=.280) 



SHuatton 10 



p„ * 0.0 and p^ * 0.0 
(r„ = .290; ^=.279) 



Situation 11 



p It * 0.0 and p^ * 0.0 
(r„ = .360; r^=.276) 



Both dependent variables l 
are related to essentially 2 
independent parts of the 
independent variable 



Both variables contribute 1 
to multivariate significance 2 
but variable 1 contributes 
more than variable 2. 



Only the first dependent 1 
variable contributes to 2 
multivariate significance. 



.707 
.707 



.654 
.610 



1.001 
-.001 



1 Standardized canonical discriminant function coefficients. 

•Example values used to illustrate a given situation and to create the discriminant function coefficienU ore given in parentheses. 
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where a variable (say, variable 2) is contributing to multivariate significance but is 
not significant when the other dependent variable (say, variable 1) is partialed 
from it 

In situations 9 and 10 both dependent variables were significantly contributing to 
overall multivariate significance. The difference between these situations is that 
in situation 9 the dependent variables are uncorrelated and in situation 10 they are 
correlated. 

In situation 11 both univariate tests were significant but only the first dependent 
variable was causing the multivariate test to be significant. This was because 
when the first dependent variable was partialed from the second dependent 
variable the second dependent variable was found to be unrelated to the 
independent variable. This result is found through the discriminant function 
coefficients where the coefficient for the first dependent variable was 1.001 and the 
coefficient for the second dependent variable was -.001 . As in the previous 
situations the discriminant function variable correlations of 1.000 and .756 
mirrored the significant univariate test results. 

Part 2 Generalization: Two Group MANOVA With Three Dependent 
Variables 

The preceding situations generalize in many respects to the two group MANOVA 
with three dependent variables. We will discuss some of the situations that differ 
from the preceding situations but that conflict with the textbook authors' assertion 
to look at the univariate tests to find out which variables caused the multivariate 
significance. 

Consider the conditions where all of the dependent variables are not significant at 
the univariate level. In this condition all of the variables may be contributing in 
some way to the multivariate significance or r ily some two of them may be 
contributing. 

In the condition where one variable is significant and two are not, it may be the 
case that the significant variable can be explained by the two variables that are not 
significant. In this case it could be that the two variables that are not significant at 
the univariate level actually cause the multivariate significance. 

In the condition where two variables are significant and one is not, it may be the 
case that the significant variable and the insignificant variable "gang up" to cause 
multivariate significance and the other significant variable does not contribute to 
multivariate significance. 

In the condition where all three dependent variables are significant it may be that 
only one, only two, cr all three dependent variables contribute to multivariate 
significance. 
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Generallzaifon: Multiple Group MANOVA With Multiple Dependent 
Variables 

As can be seen from the preceding two discussions, in a multiple group MANOVA 
with multiple dependent variables there would be a large variety of situations 
where the causes of multivariate significance would not be obvious by looking at 
the univariate test results. In these situations we suggest that researchers, armed 
with theory and a priori predictions, consider discriminant and stepwise 
discriminant analysis information to find the causes of multivariate significance. 

Educational Importance of the Study 

The results clearly indicate the ways that multivariate significance is caused 
under a variety of relationships. The results also indicate why it is improper to 
interpret univariate ANOVA results to find the causes of multivariate 
significance. Finally, the use of discriminant analysis is illustrated as a valuable 
tool in considering the contributions of dependent variables to multivariate 
significance. 

The social science literature provides many examples of studies (e.g., Brandon, 
Newton, & Hammond, 1987; Stahl & Clark, 1987) where researchers follow-up an 
overall MANOVA with univariate ANOVA's on each of the dependent variables to 
discern which of the dependent variables caused the multivariate test to be 
significant This paper illustrates the fallacy of this logic and presents educational 
researchers with illustrations of follow-up methods that help to discern what 
caused the multivariate significance. 



Elliott and Barclkowskl 



Causes of Multivariate Significance 

12 



References 

Barcikowski, R. S. (Ed.) (1983). Computer packages and research design, 
with annotations of input and output from the BMDP, SAS, SPSS, and 
SPSSX statistical packages (Vol. 1: BMDP, Vol 2: SAS, Vol 3: SPSS and 
SPSSX). Irfmhftm, MD: University Press of America. 

Brandon P. R., Newton, B. J. and Hammond, 0. W. (1987). Children's 
mathematics achievement in Hawaii: Sex differences favoring girls. 
American Educational Research Journal, 24(30), 437-461. 

Cohen, J. (1988). Statistical power analysis for the behavioral sciences, (2nd 
ecL), Hillsdale, NJ: Lawrence Erlbaum. 

Green, B. F. (1979). The two kinds of linear discriminant functions and their 
relationship. Journal of Educational Statistics, 4, 247-263. i 

Norusis, M. J. (1988). SPSS* advanced statistics guide. New York: McGraw 
Hill. 

SPSS Inc. (1988). SPSS* user's guide A (3rd ed.). New York: McGraw Hill. 

Stahl, S. A. and Clark, C. H. (1987). The effects of participatory expectations 
in classroom discussion on the learning of science vocabulary. 
American Educational Research Journal, 24(4), 641-555. 

Stevens, J. P. (1986). Applied multivariate statistics for the social sciences. 
Hillsdale, NJ: Lawrence Erlbaum. 

Tabachnick, B. G. and Pidell, L. S. (1989). Using multivariate statistics, (2nd 
ed.) New York, NY: Harper and Row Publishers. 



16 



Elliott and Btrclkowski 



Causes of Muttlvsriate Slgnmcanct 

13 



Appendix A 
Process Used To Citato Situations 1 Through 11 

The process used to create situations 1 through 11 was based on the relationship 
between a multivariate analysis for two groups (MANOVA) and the regression 
approach to this same design. In the MANOVA we assumed a large effect size of 
.80 (Cohen, 1988), a cell size of 25 (AT « 60), and multivariate significance with a 
beginning overall F statistic at approximately 3.5 (except for situation 9). 

[An F value of 3.23 or greater is necessary to have multivariate significance (p < 
.05) with 2 and 40 degrees of freedom. The F required for significance in our 
examples would actually be slightly less than 3.23 because the degrees of freedom 
for all of our MANOVA *s were 2 and 47. The F is described as "beginning" 
because during the process the F was recalculated and changed slightly, but was 
always significant at the multivariate level.] 

After selecting an F value that would be significant, we calculated the 
corresponding multiple R by use of the formula: 



-V 



2F 



(N-3) + 2F 



In all of the situations we selected values for the correlation between the dependent 
variables in the MANOVA and the correlations between the independent variable, 
x, and one or both dependent variables (denoted by 1 & 2). Using the multiple R and 
the correlation of the independent variable with the first dependent variable, rxi , 
the formula 

R*= «it + r£ (2 .i) 

was solved for the part correlation r x (2.i). 

The part correlation was then used in the formula 

r - r x2" fxl R^12 
r x(2.1) — — — - 

Vl- R r? 2 

to find t%2 the correlation between the independent variable and the second 
dependent variable. In most situations the correlation between the dependent 
variables in the multivariate analysis M r 12 was initially set equal to the 
correlation between the variables in the regression analysis, Rri2- This could be 
done because MTL2 ** generally close to Rn.2- la some situations r^2 was set to 
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zero and RH2 was found directly. 

Obtaining the proper correlations was the key to the process. Once they were found 
the formulas 



4 jx(N-2) 
N(l - 

and 



> _ / 4&(N- 2) 
" 2 "^/ N(l-r= 



N(l - 1&) 

were used to find the means Ci and C2 of the first and second, respectively, 
dependent variables for group 1. The means in group twe were set at zero for both 
dependent variables. The values of Ci and C2 were those necessary to produce 
multivariate significance (p < .05). 

The formula 

(N-2) M r 12 + N(a)(|L) 



^[N(l + (^p-2][N(\ + (ff)-2] 



was then used to find the "egression correlation ]jri2 between the two dependent 
variables. This correlation was then used to compute a new part correlation which 
in turn was used to finu the final multiple R. 

The results were checked by using the resultant summary information as input 
for the SPSSX programs REGRESSION and MANOVA. For each situation the F 
statistics and univariate test information from these programs agreed. 

Three situations used a minor variation of the preceding process. For example, in 
situation 4 we began by setting the part correlation to be larger than the correlation 
between f Jie independent variable and the second dependent variable. 

Situation 9 demanded the correlation between the two dependent variables be set to 
0.0, but the correlations between each of the dependent variables with the 
independent variable had to be large enough to lake both of the dependent 
variables significant at the univariate level. To 1 leet these conditions a beginning 
F of 4.0, instead of 3.5, was used. 
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For situation 11 it was necessary to assume the part correlation was 0.0. This 
caused r*i to be equal to the multiple R. The value of r^ was chosen large enough 
to make the second dependent variable significant at the univariate level. 
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